Skip to content

Conversation

@dishafaujdar
Copy link
Contributor

@dishafaujdar dishafaujdar commented Sep 14, 2025

feat: #4408

Summary

This PR adds automated generation of LLM discovery files to help AI systems find accurate AsyncAPI documentation and prefer v3.0 over v2.x specifications.

Changes

Files Generated

  • public/llms.txt - Small index pointing to key resources
  • public/llms-full.txt - Complete list of all AsyncAPI resources

Summary by CodeRabbit

  • New Features

    • Added two LLM-ready resource indexes: a concise llms.txt and a comprehensive llms-full.txt with categorized AsyncAPI links, recent posts, examples, and templates.
  • Documentation

    • Centralized specs, getting-started guides, docs, tools, community resources, blogs, repos and templates for improved discoverability and LLM consumption.
  • Tests

    • Added unit tests to verify generation and output of the new indexes.
  • Chores

    • Content pipeline extended to generate and publish the new LLM resource indexes.

@netlify
Copy link

netlify bot commented Sep 14, 2025

Deploy Preview for asyncapi-website ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 56e453e
🔍 Latest deploy log https://app.netlify.com/projects/asyncapi-website/deploys/68fd10d66c008200081459a2
😎 Deploy Preview https://deploy-preview-4416--asyncapi-website.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 14, 2025

Walkthrough

Adds two new LLMS generator scripts (llms.txt and llms-full.txt), integrates them into the build pipeline, commits their generated public text artifacts, and adds unit tests for both generators.

Changes

Cohort / File(s) Summary
LLMS full generator
scripts/build-llms-full.ts
New exported buildLlmsFull(): Promise<void> that dynamic-imports ../config/posts.json, assembles llms-full.txt with multiple categorized sections, deduplicates URLs/slugs (throws on duplicates), prunes docs → 50 and tools → 20, sorts/limits recent posts (20), writes ./public/llms-full.txt (UTF-8), and logs totals.
LLMS simple generator
scripts/build-llms.ts
New exported buildLlmsTxt(): Promise<void> that builds an llms.txt document (llmstxt.org style) with title, summary, and categorized links, writes ./public/llms.txt (UTF-8), and logs success.
Pipeline integration
scripts/index.ts
Imports and invokes buildLlmsTxt() and buildLlmsFull() from start() after RSS generation, running them sequentially in the content build flow.
Public artifacts
public/llms.txt, public/llms-full.txt
New generated text files added as public resources containing organized AsyncAPI links and references for LLM consumption.
Tests
tests/scripts/build-llms.test.ts, tests/scripts/build-llms-full.test.ts
New unit tests that mock fs/promises.writeFile; build-llms-full test also mocks posts.json. Tests assert write calls to ./public/llms.txt and ./public/llms-full.txt and verify key content/invocation.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Index as scripts/index.ts
  participant BuildTxt as buildLlmsTxt()
  participant BuildFull as buildLlmsFull()
  participant Posts as ../config/posts.json
  participant FS as File System

  Index->>BuildTxt: call buildLlmsTxt()
  alt generate llms.txt
    BuildTxt->>FS: write ./public/llms.txt
    FS-->>BuildTxt: OK
  end
  BuildTxt-->>Index: done

  Index->>BuildFull: call buildLlmsFull()
  alt load posts.json
    BuildFull->>Posts: dynamic import posts.json
    Posts-->>BuildFull: posts data
  end
  BuildFull->>BuildFull: assemble sections, dedupe, prune, sort
  BuildFull->>FS: write ./public/llms-full.txt
  FS-->>BuildFull: OK
  BuildFull-->>Index: done
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • Inspect duplicate detection and error message formatting in scripts/build-llms-full.ts.
  • Verify dynamic import of ../config/posts.json works in runtime and test environments.
  • Check pruning limits (docs → 50, tools → 20), recent posts sorting/limit, and UTF-8 write behavior.
  • Review tests' mocking of fs/promises.writeFile and module reset/restore logic.

Possibly related PRs

Suggested labels

ready-to-merge

Suggested reviewers

  • derberg
  • anshgoyalevil
  • sambhavgupta0705
  • akshatnema
  • devilkiller-ag
  • Mayaleeeee

Poem

I nibble links and stitch each line,
I hop through posts and sort by time,
I prune the lists and chase the best,
Then write a file — a rabbit's zest. 🥕

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "feat: add llms files" is directly related to the main changeset, which adds automated generation of LLMS discovery files (public/llms.txt and public/llms-full.txt) along with their build scripts. The title accurately conveys the primary change—that LLMS-related files are being added—and is concise and clear without unnecessary noise or vague terminology. While it could be more descriptive about the purpose (LLM discovery for AI systems) or the automation aspect, it sufficiently summarizes the core change and would help teammates understand what was modified when scanning commit history.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (2)
components/Calendar.tsx (1)

54-54: Trim trailing whitespace.

Minor tidy-up.

-              </div>  
+              </div>
scripts/index.ts (1)

37-38: Consider isolating LLMS generation failures from the main build.

Optional guard so docs build isn’t blocked by non-critical LLMS outputs.

-  await buildLlmsTxt();
-  await buildLlmsFull();
+  try {
+    await buildLlmsTxt();
+    await buildLlmsFull();
+  } catch (err) {
+    console.error('LLMS build failed:', err);
+  }
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1b423a5 and 6f3e154.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (4)
  • components/Calendar.tsx (2 hunks)
  • scripts/build-llms-full.ts (1 hunks)
  • scripts/build-llms.ts (1 hunks)
  • scripts/index.ts (2 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2024-11-01T09:35:23.912Z
Learnt from: akshatnema
PR: asyncapi/website#3101
File: scripts/build-rss.js:4-6
Timestamp: 2024-11-01T09:35:23.912Z
Learning: Converting `getAllPosts()` to asynchronous in `scripts/build-rss.js` causes the existing build system to fail.

Applied to files:

  • scripts/build-llms-full.ts
  • scripts/index.ts
📚 Learning: 2024-11-10T18:16:22.273Z
Learnt from: vishvamsinh28
PR: asyncapi/website#3284
File: scripts/build-post-list.js:49-53
Timestamp: 2024-11-10T18:16:22.273Z
Learning: In `scripts/build-post-list.js`, the `walkDirectories` function depends on sequential execution because it mutates `resultObj`. Parallelizing the loop with `Promise.all` causes tests to fail.

Applied to files:

  • scripts/index.ts
🧬 Code graph analysis (2)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
scripts/index.ts (2)
scripts/build-llms.ts (1)
  • buildLlmsTxt (15-56)
scripts/build-llms-full.ts (1)
  • buildLlmsFull (28-144)
🪛 GitHub Actions: PR testing - if Node project
scripts/build-llms-full.ts

[error] 10-10: ModuleNotFoundError: Cannot find module '../config/posts.json' from 'scripts/build-llms-full.ts' during 'npm test --if-present'.

🪛 Biome (2.1.2)
components/Calendar.tsx

[error] 35-35: Expected an expression but instead found '<<'.

Expected an expression here.

(parse)


[error] 35-35: Expected an expression but instead found '<<'.

Expected an expression here.

(parse)


[error] 36-36: expected , but instead found className

Remove className

(parse)


[error] 36-36: expected , but instead found =

Remove =

(parse)


[error] 36-36: expected , but instead found twMerge

Remove twMerge

(parse)


[error] 36-36: expected , but instead found (

Remove (

(parse)


[error] 36-36: expected ) but instead found ,

Remove ,

(parse)


[error] 36-36: expected , but instead found )

Remove )

(parse)


[error] 36-37: Expected a JSX attribute but instead found '======='.

Expected a JSX attribute here.

(parse)


[error] 38-38: expected > but instead found <

Remove <

(parse)


[error] 43-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)

🪛 GitHub Actions: Run Cypress E2E Tests
components/Calendar.tsx

[error] 35-74: Merge conflict marker encountered in Calendar.tsx. Remove conflict markers (<<<<<<< HEAD, =======, >>>>>>> upstream/master) and resolve the merge conflict.

⏰ Context from checks skipped due to timeout of 180000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Lighthouse CI
🔇 Additional comments (1)
scripts/build-llms.ts (1)

32-32: No change required — default branch is master.
Default branch is "master" and the spec file is at spec/asyncapi.md, so the existing blob/master/spec/asyncapi.md link is correct.

Comment on lines 35 to 41
<<<<<<< HEAD
<div className={twMerge('overflow-hidden rounded-md border border-gray-200 bg-white p-4 h-full', className)}>
=======
<div
className={twMerge(
'overflow-hidden rounded-md border border-gray-200 bg-white p-4 h-full flex flex-col gap-2',
className
)}
>
>>>>>>> upstream/master
<Heading level={HeadingLevel.h2} typeStyle={HeadingTypeStyle.mdSemibold}>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Resolve merge conflict markers; keep the flex wrapper.

Conflict markers break the build and E2E. Prefer the upstream/master wrapper to preserve layout.

Apply this diff:

-<<<<<<< HEAD
-    <div className={twMerge('overflow-hidden rounded-md border border-gray-200 bg-white p-4 h-full', className)}>
-=======
     <div
       className={twMerge(
         'overflow-hidden rounded-md border border-gray-200 bg-white p-4 h-full flex flex-col gap-2',
         className
       )}
     >
->>>>>>> upstream/master

Run to ensure no other conflict markers remain:


🏁 Script executed:

#!/bin/bash
rg -n '^(<<<<<<<|=======|>>>>>>>)' -S

Length of output: 803


Resolve merge conflict markers; keep upstream/master flex wrapper

  • components/Calendar.tsx — remove the merge markers at lines 35–44 and keep the upstream/master div (the twMerge variant with 'flex flex-col gap-2') so layout is preserved.
  • rg also matched lines starting with '=======' in: markdown/blog/asyncapi-and-apicurio-for-asynchronous-apis.md (lines 30,156,186,211) and LICENSE-docs (lines 3,55,376) — inspect and remove any true conflict separators.
  • After fixes, run: rg -n '^(<<<<<<<|=======|>>>>>>>)' -S to confirm no markers remain.
🧰 Tools
🪛 Biome (2.1.2)

[error] 35-35: Expected an expression but instead found '<<'.

Expected an expression here.

(parse)


[error] 35-35: Expected an expression but instead found '<<'.

Expected an expression here.

(parse)


[error] 36-36: expected , but instead found className

Remove className

(parse)


[error] 36-36: expected , but instead found =

Remove =

(parse)


[error] 36-36: expected , but instead found twMerge

Remove twMerge

(parse)


[error] 36-36: expected , but instead found (

Remove (

(parse)


[error] 36-36: expected ) but instead found ,

Remove ,

(parse)


[error] 36-36: expected , but instead found )

Remove )

(parse)


[error] 36-37: Expected a JSX attribute but instead found '======='.

Expected a JSX attribute here.

(parse)


[error] 38-38: expected > but instead found <

Remove <

(parse)


[error] 43-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)


[error] 44-44: Unexpected token. Did you mean {'>'} or &gt;?

(parse)

🪛 GitHub Actions: Run Cypress E2E Tests

[error] 35-74: Merge conflict marker encountered in Calendar.tsx. Remove conflict markers (<<<<<<< HEAD, =======, >>>>>>> upstream/master) and resolve the merge conflict.

🤖 Prompt for AI Agents
In components/Calendar.tsx around lines 35–45 there are unresolved Git conflict
markers; remove the conflict markers and keep the upstream/master div variant
that includes twMerge('overflow-hidden rounded-md border border-gray-200
bg-white p-4 h-full flex flex-col gap-2', className) so the flex layout is
preserved, deleting the HEAD version and all <<<<<<<, =======, >>>>>>> lines;
also inspect the other files flagged by rg
(markdown/blog/asyncapi-and-apicurio-for-asynchronous-apis.md at lines
~30,156,186,211 and LICENSE-docs at lines ~3,55,376) and remove any true
conflict separators there as well; after edits run rg -n
'^(<<<<<<<|=======|>>>>>>>)' -S to confirm no conflict markers remain.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (3)
scripts/build-llms-full.ts (3)

9-12: Fix JSON import: replace dynamic JSON import with fs+path (avoids ESM JSON import and bundler pitfalls).

Dynamic import('../config/posts.json') commonly fails in CI/tests (needs assert { type: 'json' }, bundler hooks, or experimental flags). Read via FS with an absolute path.

Apply:

-import fs from 'fs';
+import { promises as fs } from 'node:fs';
+import { dirname, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
@@
-async function getAllPosts(): Promise<Result> {
-  const posts = (await import('../config/posts.json')).default as Result;
-  return posts;
-}
+async function getAllPosts(): Promise<Result> {
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const jsonPath = resolve(__dirname, '../config/posts.json');
+  const raw = await fs.readFile(jsonPath, 'utf8');
+  return JSON.parse(raw) as Result;
+}

Also applies to: 1-2


135-144: Fix “Total URLs” undercount; write asynchronously and ensure ./public exists; make output path CWD‑safe.

Current count is computed before the final block and sync write may race/build stall; also missing mkdir. Compute after assembling, append, mkdir -p, and write via async API using a path relative to this file.

-# END OF RESOURCE LIST
-# Total URLs: ${content.split('\n').filter(line => line.includes('http')).length}
-`;
-
-  // Write the file
-  fs.writeFileSync('./public/llms-full.txt', content, 'utf8');
-  
-  const urlCount = content.split('\n').filter(line => line.includes('http')).length;
-  console.log(`✅ llms-full.txt generated successfully with ${urlCount} URLs at ./public/llms-full.txt`);
+# END OF RESOURCE LIST
+`;
+
+  // Compute final URL count and append
+  const urlCount = content
+    .split('\n')
+    .filter((line) => /^https?:\/\//.test(line.trim()))
+    .length;
+  content += `# Total URLs: ${urlCount}\n`;
+
+  // Ensure output dir and write (path resolved relative to this file)
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const outDir = resolve(__dirname, '../public');
+  const outFile = resolve(outDir, 'llms-full.txt');
+  await fs.mkdir(outDir, { recursive: true });
+  await fs.writeFile(outFile, content, 'utf8');
+  console.log(`✅ llms-full.txt generated successfully with ${urlCount} URLs at ${outFile}`);

Also applies to: 60-60


91-103: Guard non-array post types and skip docsTree.

Prevents accidental inclusion of nav trees/objects.

-  Object.keys(posts).forEach(postType => {
-    if (!['docs', 'blog', 'about'].includes(postType)) {
-      const postArray = (posts as any)[postType];
-      if (postArray && postArray.length > 0) {
+  Object.keys(posts).forEach((postType) => {
+    if (!['docs', 'blog', 'about', 'docsTree'].includes(postType)) {
+      const postArray = (posts as any)[postType];
+      if (Array.isArray(postArray) && postArray.length > 0) {
         const sectionTitle = postType.charAt(0).toUpperCase() + postType.slice(1);
         content += `\n## ${sectionTitle}\n`;
         postArray.forEach((item: any) => {
           content += `${base}${item.slug} - ${item.title}\n`;
         });
       }
     }
   });
🧹 Nitpick comments (4)
scripts/build-llms-full.ts (4)

72-81: Harden blog sorting for missing/invalid dates.

new Date(x).getTime() can be NaN, making comparator unstable.

-    posts.blog
-      .sort((a: any, b: any) => new Date(b.date).getTime() - new Date(a.date).getTime())
+    posts.blog
+      .sort((a: any, b: any) => {
+        const tb = Date.parse(b?.date ?? '');
+        const ta = Date.parse(a?.date ?? '');
+        return (isNaN(tb) ? 0 : tb) - (isNaN(ta) ? 0 : ta); // desc
+      })
       .forEach((post: any) => {
-        const date = post.date ? ` (${post.date.split('T')[0]})` : '';
+        const date =
+          post?.date && !isNaN(Date.parse(post.date))
+            ? ` (${new Date(post.date).toISOString().split('T')[0]})`
+            : ''
         content += `${base}${post.slug} - ${post.title}${date}\n`;
       });

31-31: Make base URL configurable (works in previews and local runs).

-  const base = 'https://www.asyncapi.com';
+  const base = (process.env.SITE_URL ?? 'https://www.asyncapi.com').replace(/\/$/, '');

127-129: Correct v2 vs v3 note for accuracy.

v2 already has channels; the change is how operations are modeled.

-# v3.0 uses 'channels' and 'operations' objects instead of v2's inline channel definitions
+# In v2, operations live inline on each channel item via `publish`/`subscribe`.
+# In v3.0, operations are first-class under a top-level `operations` object and channels reference them.

65-69: Optional: remove any by using the Details type for stronger checks.

-      .forEach((doc: any) => {
+      .forEach((doc: { slug: string; title: string }) => {
         content += `${base}${doc.slug} - ${doc.title}\n`;
       });
@@
-      .forEach((post: any) => {
+      .forEach((post: { slug: string; title: string; date?: string }) => {
         const date = post.date ? ` (${post.date.split('T')[0]})` : '';
         content += `${base}${post.slug} - ${post.title}${date}\n`;
       });
@@
-    posts.about.forEach((page: any) => {
+    posts.about.forEach((page: { slug: string; title: string }) => {
       content += `${base}${page.slug} - ${page.title}\n`;
     });
@@
-        postArray.forEach((item: any) => {
+        postArray.forEach((item: { slug: string; title: string }) => {
           content += `${base}${item.slug} - ${item.title}\n`;
         });

Also applies to: 75-80, 86-88, 98-100

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6f3e154 and 1b06069.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (1)
  • scripts/build-llms-full.ts (1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2024-11-01T09:35:23.912Z
Learnt from: akshatnema
PR: asyncapi/website#3101
File: scripts/build-rss.js:4-6
Timestamp: 2024-11-01T09:35:23.912Z
Learning: Converting `getAllPosts()` to asynchronous in `scripts/build-rss.js` causes the existing build system to fail.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2025-01-18T08:44:43.614Z
Learnt from: akshatnema
PR: asyncapi/website#3423
File: tests/index.test.js:2-7
Timestamp: 2025-01-18T08:44:43.614Z
Learning: In the AsyncAPI website project, JavaScript test files must include the .ts extension when importing TypeScript files (e.g., `require('../scripts/build-rss.ts')`). This is enforced by the project's configuration which uses `moduleResolution: "bundler"` in tsconfig.json and TypeScript-aware ESLint plugins. The .ts extensions are required even though the files are imported using CommonJS require statements.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2024-11-10T18:16:22.273Z
Learnt from: vishvamsinh28
PR: asyncapi/website#3284
File: scripts/build-post-list.js:49-53
Timestamp: 2024-11-10T18:16:22.273Z
Learning: In `scripts/build-post-list.js`, the `walkDirectories` function depends on sequential execution because it mutates `resultObj`. Parallelizing the loop with `Promise.all` causes tests to fail.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2025-01-18T08:44:43.614Z
Learnt from: akshatnema
PR: asyncapi/website#3423
File: tests/index.test.js:2-7
Timestamp: 2025-01-18T08:44:43.614Z
Learning: In the AsyncAPI website project, JavaScript test files must include the .ts extension when importing TypeScript files (e.g., `require('../scripts/build-rss.ts')`). This is a project-specific requirement enforced by the linting rules and build setup, even though it differs from typical Node.js behavior.

Applied to files:

  • scripts/build-llms-full.ts
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
⏰ Context from checks skipped due to timeout of 180000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Lighthouse CI
🔇 Additional comments (2)
scripts/build-llms-full.ts (2)

28-33: No action required — buildLlmsFull is awaited in scripts/index.ts.
Found await buildLlmsFull() at scripts/index.ts:38; no changes needed.


41-46: Verify external URLs/branches and confirm definitions endpoints

File: scripts/build-llms-full.ts (lines 41–46)

  • GitHub spec link uses blob/master — the file is reachable at that path; confirm the repo default branch (master vs main) and update the link if the default is now main. (github.com)
  • Definitions JSON endpoints under https://www.asyncapi.com/definitions/... (3.0.0, 2.6.0, 2.5.0, 2.4.0) could not be fetched by the automated check — run curl/HEAD from CI or locally to confirm they exist or redirect. ()
  • Case studies path is /casestudies (not /case‑studies). (asyncapi.com)
  • Slack entry asyncapi.slack.com is reachable. (asyncapi.slack.com)
  • v3 and v2 spec reference pages exist at /docs/reference/specification/v3.0.0 and /docs/reference/specification/v2.6.0. (asyncapi.com)

Also applies to lines: 47–55, 56–59, 116–121, 123–124, 130–134.

@dishafaujdar dishafaujdar changed the title Feature/add llms files feat: Feature/add llms files Sep 14, 2025
@dishafaujdar dishafaujdar changed the title feat: Feature/add llms files feat: add llms files Sep 14, 2025
@asyncapi-bot
Copy link
Contributor

asyncapi-bot commented Sep 14, 2025

⚡️ Lighthouse report for the changes in this PR:

Category Score
🔴 Performance 36
🟢 Accessibility 98
🟢 Best practices 92
🟢 SEO 100
🔴 PWA 33

Lighthouse ran on https://deploy-preview-4416--asyncapi-website.netlify.app/

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
components/Calendar.tsx (3)

50-50: Strip trailing whitespace to satisfy linters

Extraneous spaces after the closing div can trip formatters.

-              </div>  
+              </div>

32-33: Potential runtime crash when eventsData is undefined

eventsData?.filter(...).length can throw because .length is accessed on undefined when eventsData is absent. Use .some() with nullish coalescing.

-  const eventsExist = eventsData?.filter((event: IEvent) => moment(event.date).isAfter(currentDate)).length > 0;
+  const eventsExist = eventsData?.some((event: IEvent) => moment(event.date).isAfter(currentDate)) ?? false;

46-46: Avoid array index as React key

Use a stable, unique key to prevent reconciliation bugs.

-          <li key={index} data-testid='Calendar-list-item'>
+          <li key={event.url || `${event.date}-${index}`} data-testid='Calendar-list-item'>
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1b06069 and 08841f2.

📒 Files selected for processing (1)
  • components/Calendar.tsx (1 hunks)
⏰ Context from checks skipped due to timeout of 180000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Lighthouse CI
🔇 Additional comments (1)
components/Calendar.tsx (1)

35-40: Merge markers resolved and flex wrapper preserved — LGTM; confirm no real conflict markers remain

  • rg matched long '=' lines at: markdown/blog/asyncapi-and-apicurio-for-asynchronous-apis.md:30,156,186,211 and LICENSE-docs:3,55,376 — these are repeated '=' lines (matched by the previous regex), not explicit '<<<<<<<'/'>>>>>>>' markers.
  • Run this targeted check to be sure: rg -n '^(<<<<<<<|>>>>>>>)' -S — if it returns nothing, mark resolved.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
scripts/build-llms.ts (1)

53-56: Async FS + mkdir — resolved.

You switched to async fs and ensure ./public exists. This addresses the earlier ENOENT/blocking I/O concern.

scripts/build-llms-full.ts (1)

11-17: Good: robust FS + absolute path for posts.json.

Switching from dynamic import to fs+path fixes ModuleNotFoundError in tests/build.

🧹 Nitpick comments (7)
scripts/build-llms.ts (3)

18-18: Allow base URL override (keep canonical as default).

Make the base host configurable for previews/self-hosts while defaulting to asyncapi.com.

Apply:

-  const base = 'https://www.asyncapi.com';
+  const base = (process.env.SITE_URL || 'https://www.asyncapi.com').replace(/\/$/, '');

29-33: Prefer docs site for spec reference (GitHub branch can drift).

Add the v3 docs reference (keep GitHub as secondary if you want).

 ## Specifications & Schemas  
 ${base}/definitions/3.0.0/asyncapi.json - AsyncAPI 3.0.0 JSON Schema
 ${base}/definitions/2.6.0/asyncapi.json - AsyncAPI 2.6.0 JSON Schema
-https://github.com/asyncapi/spec/blob/master/spec/asyncapi.md - Latest specification
+${base}/docs/reference/specification/v3.0.0 - Latest specification ( docs )
+https://github.com/asyncapi/spec - Specification source ( GitHub )

1-1: Minor: use node:fs/promises and absolute output path (CWD‑safe).

Avoid CWD assumptions and use the promises module directly.

-import fs from 'fs';
+import { mkdir, writeFile } from 'node:fs/promises';
+import { resolve, dirname } from 'node:path';
+import { fileURLToPath } from 'node:url';
@@
-  await fs.promises.mkdir('./public', { recursive: true });
-  await fs.promises.writeFile('./public/llms.txt', llmsContent, 'utf8');
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const outDir = resolve(__dirname, '../public');
+  const outFile = resolve(outDir, 'llms.txt');
+  await mkdir(outDir, { recursive: true });
+  await writeFile(outFile, llmsContent, 'utf8');

Also applies to: 54-55

scripts/build-llms-full.ts (4)

79-85: Guard against invalid/missing blog dates in sort.

If a post lacks a valid date, the comparator can return NaN leading to unstable order.

-      .sort((a: any, b: any) => new Date(b.date).getTime() - new Date(a.date).getTime())
+      .sort((a: any, b: any) => {
+        const ta = Number.isFinite(new Date(a?.date ?? '').getTime()) ? new Date(a.date).getTime() : 0;
+        const tb = Number.isFinite(new Date(b?.date ?? '').getTime()) ? new Date(b.date).getTime() : 0;
+        return tb - ta;
+      })

110-120: Use stable join link for Slack.

Workspaces URLs can require auth; prefer the public join URL on the site.

-https://asyncapi.slack.com - Official Slack community
+${base}/slack - Official Slack community

143-146: Count URLs precisely (not just lines containing “http”).

This avoids under/over‑count and is resilient if multiple URLs appear on a line.

-  const urlCount = content.split('\n').filter((line) => line.includes('http')).length;
+  const urlCount = (content.match(/https?:\/\/\S+/g) || []).length;

147-150: CWD‑independent output path.

Resolve ./public relative to this script to avoid surprises in different runners.

-  await fs.promises.mkdir('./public', { recursive: true });
-  await fs.promises.writeFile('./public/llms-full.txt', content, 'utf8');
-  console.log(`✅ llms-full.txt generated successfully with ${urlCount} URLs at ./public/llms-full.txt`);
+  const outDir = resolve(dirname(fileURLToPath(import.meta.url)), '../public');
+  const outFile = resolve(outDir, 'llms-full.txt');
+  await fs.promises.mkdir(outDir, { recursive: true });
+  await fs.promises.writeFile(outFile, content, 'utf8');
+  console.log(`✅ llms-full.txt generated successfully with ${urlCount} URLs at ${outFile}`);
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 08841f2 and 4ca90e4.

📒 Files selected for processing (2)
  • scripts/build-llms-full.ts (1 hunks)
  • scripts/build-llms.ts (1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2024-11-01T09:35:23.912Z
Learnt from: akshatnema
PR: asyncapi/website#3101
File: scripts/build-rss.js:4-6
Timestamp: 2024-11-01T09:35:23.912Z
Learning: Converting `getAllPosts()` to asynchronous in `scripts/build-rss.js` causes the existing build system to fail.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2025-01-18T08:44:43.614Z
Learnt from: akshatnema
PR: asyncapi/website#3423
File: tests/index.test.js:2-7
Timestamp: 2025-01-18T08:44:43.614Z
Learning: In the AsyncAPI website project, JavaScript test files must include the .ts extension when importing TypeScript files (e.g., `require('../scripts/build-rss.ts')`). This is enforced by the project's configuration which uses `moduleResolution: "bundler"` in tsconfig.json and TypeScript-aware ESLint plugins. The .ts extensions are required even though the files are imported using CommonJS require statements.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2024-11-10T18:16:22.273Z
Learnt from: vishvamsinh28
PR: asyncapi/website#3284
File: scripts/build-post-list.js:49-53
Timestamp: 2024-11-10T18:16:22.273Z
Learning: In `scripts/build-post-list.js`, the `walkDirectories` function depends on sequential execution because it mutates `resultObj`. Parallelizing the loop with `Promise.all` causes tests to fail.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2025-01-18T08:44:43.614Z
Learnt from: akshatnema
PR: asyncapi/website#3423
File: tests/index.test.js:2-7
Timestamp: 2025-01-18T08:44:43.614Z
Learning: In the AsyncAPI website project, JavaScript test files must include the .ts extension when importing TypeScript files (e.g., `require('../scripts/build-rss.ts')`). This is a project-specific requirement enforced by the linting rules and build setup, even though it differs from typical Node.js behavior.

Applied to files:

  • scripts/build-llms-full.ts
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
🔇 Additional comments (4)
scripts/build-llms-full.ts (3)

96-108: Nice: skip non‑array post types (docsTree).

Prevents accidental inclusion of nav trees/objects. LGTM.


33-37: Confirmed — both buildLlmsTxt() and buildLlmsFull() are awaited in the build orchestrator.
scripts/index.ts calls await buildLlmsTxt(); and await buildLlmsFull(); (lines 37–38).


127-130: Replace invalid repo link; spec uses “master”

File: scripts/build-llms-full.ts lines 127-130 — the URL github.com/asyncapi/asyncapi/tree/master/examples is incorrect (no repo named "asyncapi" in the org). (github.com)

Examples are in asyncapi/spec/tree/master/examples and the spec repo’s default branch is master. Update the first URL to asyncapi/spec/tree/master/examples or confirm the intended repo. (github.com)

Likely an incorrect or invalid review comment.

scripts/build-llms.ts (1)

41-47: Verified: 'casestudies' and 'tools' routes exist.
Found: pages/casestudies/index.tsx, pages/casestudies/[id].tsx, pages/tools/index.tsx (also pages/[lang]/tools). No renaming required.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (11)
scripts/build-llms-full.ts (11)

1-4: Prefer node: specifiers and fs.promises for clarity and ESM correctness.

Use built‑in module specifiers and import promises API directly; simplifies calls and avoids CJS interop edge cases.

-import fs from 'fs';
-import type { Result } from '@/types/scripts/build-posts-list';
-import { dirname, resolve } from 'path';
+import { promises as fs } from 'node:fs';
+import type { Result } from '@/types/scripts/build-posts-list';
+import { dirname, resolve } from 'node:path';
 import { fileURLToPath } from 'url';

And update below:

-  const raw = await fs.promises.readFile(jsonPath, 'utf8');
+  const raw = await fs.readFile(jsonPath, 'utf8');
-  await fs.promises.mkdir('./public', { recursive: true });
-  await fs.promises.writeFile('./public/llms-full.txt', content, 'utf8');
+  await fs.mkdir(outDir, { recursive: true });
+  await fs.writeFile(outPath, content, 'utf8');

11-17: Harden read/parse with actionable error messages.

Wrap IO/JSON parse to aid CI debugging.

 async function getAllPosts(): Promise<Result> {
   const __filename = fileURLToPath(import.meta.url);
   const __dirname = dirname(__filename);
   const jsonPath = resolve(__dirname, '../config/posts.json');
-  const raw = await fs.promises.readFile(jsonPath, 'utf8');
-  return JSON.parse(raw) as Result;
+  try {
+    const raw = await fs.readFile(jsonPath, 'utf8');
+    return JSON.parse(raw) as Result;
+  } catch (err) {
+    const msg = err instanceof Error ? err.message : String(err);
+    throw new Error(`Failed to read/parse posts.json at ${jsonPath}: ${msg}`);
+  }
 }

Does this script also run from your compiled output (if you compile scripts), or only via tsx/ts-node? If compiled, confirm that ../config/posts.json is available relative to the emitted scripts.


33-38: Make base URL configurable to support previews/staging.

Allow override via env while defaulting to production.

-export async function buildLlmsFull(): Promise<void> {
+export async function buildLlmsFull(): Promise<void> {
   console.log('Building llms-full.txt...');
   
-  const base = 'https://www.asyncapi.com';
+  const base = (process.env.SITE_URL || 'https://www.asyncapi.com').replace(/\/$/, '');
   const posts = await getAllPosts();

Set SITE_URL in Netlify/Vercel preview so links match the deploy-preview domain if desired.


45-51: Avoid hard‑coding “master” branch in GitHub links.

Use HEAD to be branch‑agnostic.

-https://github.com/asyncapi/spec/blob/master/spec/asyncapi.md - Latest specification source
+https://github.com/asyncapi/spec/blob/HEAD/spec/asyncapi.md - Latest specification source

67-75: Normalize URLs when concatenating base + slug.

Guard against missing/duplicate slashes by using URL().

-    posts.docs
+    posts.docs
       .sort((a, b) => (a.slug ?? '').localeCompare(b.slug ?? ''))
       .forEach((doc: any) => {
-        content += `${base}${doc.slug} - ${doc.title}\n`;
+        const href = new URL(doc.slug ?? '/', base).toString();
+        content += `${href} - ${doc.title}\n`;
       });

77-86: Handle missing/invalid dates in blog sort to avoid NaN comparator.

Current comparator can return NaN leading to unstable ordering.

-    posts.blog
-      .sort((a: any, b: any) => new Date(b.date).getTime() - new Date(a.date).getTime())
+    const toEpoch = (d?: string) => (d ? Date.parse(d) || 0 : 0);
+    posts.blog
+      .sort((a: any, b: any) => toEpoch(b.date) - toEpoch(a.date))
       .forEach((post: any) => {
-        const date = post.date ? ` (${post.date.split('T')[0]})` : '';
-        content += `${base}${post.slug} - ${post.title}${date}\n`;
+        const href = new URL(post.slug ?? '/', base).toString();
+        const date = post.date ? ` (${(post.date as string).split('T')[0]})` : '';
+        content += `${href} - ${post.title}${date}\n`;
       });

88-94: Stabilize About page order.

Sort for deterministic output.

-    posts.about.forEach((page: any) => {
-      content += `${base}${page.slug} - ${page.title}\n`;
-    });
+    posts.about
+      .sort((a: any, b: any) => (a.slug ?? '').localeCompare(b.slug ?? ''))
+      .forEach((page: any) => {
+        const href = new URL(page.slug ?? '/', base).toString();
+        content += `${href} - ${page.title}\n`;
+      });

96-108: Dynamic sections: type‑safety and deterministic order.

Avoid any and keep sorting consistent.

-  Object.keys(posts).forEach((postType) => {
+  (Object.keys(posts) as string[]).forEach((postType) => {
     if (!['docs', 'blog', 'about', 'docsTree'].includes(postType)) {
-      const postArray = (posts as any)[postType];
-      if (Array.isArray(postArray) && postArray.length > 0) {
+      const postArray = (posts as any)[postType];
+      if (Array.isArray(postArray) && postArray.length > 0) {
         const sectionTitle = postType.charAt(0).toUpperCase() + postType.slice(1);
         content += `\n## ${sectionTitle}\n`;
-        postArray.forEach((item: any) => {
-          content += `${base}${item.slug} - ${item.title}\n`;
-        });
+        postArray
+          .sort((a: any, b: any) => (a.slug ?? '').localeCompare(b.slug ?? ''))
+          .forEach((item: any) => {
+            const href = new URL(item.slug ?? '/', base).toString();
+            content += `${href} - ${item.title}\n`;
+          });
       }
     }
   });

127-130: Avoid hard‑coded “master” in example links.

Prefer HEAD to survive default‑branch changes.

-https://github.com/asyncapi/asyncapi/tree/master/examples - Official AsyncAPI document examples
-https://github.com/asyncapi/spec/tree/master/examples - Specification examples
+https://github.com/asyncapi/asyncapi/tree/HEAD/examples - Official AsyncAPI document examples
+https://github.com/asyncapi/spec/tree/HEAD/examples - Specification examples

Confirm the first link actually has an examples folder; if not, drop it and keep only the spec repo.


135-139: Use a public Slack join URL instead of workspace URL.

Direct workspace URL often requires being logged in.

-https://asyncapi.slack.com - Official Slack community
+https://www.asyncapi.com/slack - Official Slack community

143-151: Robust URL counting and path resolution for output.

Count actual URLs (regex) and write using an absolute path relative to this script to avoid CWD issues.

-  // Calculate URL count after all content is assembled
-  const urlCount = content.split('\n').filter((line) => line.includes('http')).length;
+  // Calculate URL count after all content is assembled
+  const urlCount = (content.match(/\bhttps?:\/\/\S+/g) ?? []).length;
   content += `# Total URLs: ${urlCount}\n`;
 
-  // Ensure output dir exists and write the file
-  await fs.promises.mkdir('./public', { recursive: true });
-  await fs.promises.writeFile('./public/llms-full.txt', content, 'utf8');
-  console.log(`✅ llms-full.txt generated successfully with ${urlCount} URLs at ./public/llms-full.txt`);
+  // Ensure output dir exists and write the file (relative to repo, not CWD)
+  const outDir = resolve(dirname(fileURLToPath(import.meta.url)), '../public');
+  const outPath = resolve(outDir, 'llms-full.txt');
+  await fs.mkdir(outDir, { recursive: true });
+  await fs.writeFile(outPath, content, 'utf8');
+  console.log(`✅ llms-full.txt generated successfully with ${urlCount} URLs at ${outPath}`);
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4ca90e4 and 30cdf5b.

📒 Files selected for processing (1)
  • scripts/build-llms-full.ts (1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2024-11-01T09:35:23.912Z
Learnt from: akshatnema
PR: asyncapi/website#3101
File: scripts/build-rss.js:4-6
Timestamp: 2024-11-01T09:35:23.912Z
Learning: Converting `getAllPosts()` to asynchronous in `scripts/build-rss.js` causes the existing build system to fail.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2025-01-18T08:44:43.614Z
Learnt from: akshatnema
PR: asyncapi/website#3423
File: tests/index.test.js:2-7
Timestamp: 2025-01-18T08:44:43.614Z
Learning: In the AsyncAPI website project, JavaScript test files must include the .ts extension when importing TypeScript files (e.g., `require('../scripts/build-rss.ts')`). This is enforced by the project's configuration which uses `moduleResolution: "bundler"` in tsconfig.json and TypeScript-aware ESLint plugins. The .ts extensions are required even though the files are imported using CommonJS require statements.

Applied to files:

  • scripts/build-llms-full.ts
📚 Learning: 2024-11-10T18:16:22.273Z
Learnt from: vishvamsinh28
PR: asyncapi/website#3284
File: scripts/build-post-list.js:49-53
Timestamp: 2024-11-10T18:16:22.273Z
Learning: In `scripts/build-post-list.js`, the `walkDirectories` function depends on sequential execution because it mutates `resultObj`. Parallelizing the loop with `Promise.all` causes tests to fail.

Applied to files:

  • scripts/build-llms-full.ts
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)

Comment on lines 110 to 119
// Add GitHub repositories and tools
content += `\n## GitHub Repositories & Tools
https://github.com/asyncapi/asyncapi - Main AsyncAPI specification repository
https://github.com/asyncapi/generator - AsyncAPI code/docs generator
https://github.com/asyncapi/parser - AsyncAPI document parser
https://github.com/asyncapi/modelina - AsyncAPI model/type generator
https://github.com/asyncapi/cli - AsyncAPI CLI tool
https://github.com/asyncapi/studio - AsyncAPI visual editor
https://github.com/asyncapi/website - This website's source code
https://github.com/asyncapi/community - Community resources and governance
## Migration Guides & Best Practices
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Verify all GitHub repo links (some repos moved/renamed/archived).

Run a quick status check to catch 404s and branch issues.


🏁 Script executed:

#!/bin/bash
set -euo pipefail
urls=(
  "https://github.com/asyncapi/asyncapi"
  "https://github.com/asyncapi/generator"
  "https://github.com/asyncapi/parser"
  "https://github.com/asyncapi/modelina"
  "https://github.com/asyncapi/cli"
  "https://github.com/asyncapi/studio"
  "https://github.com/asyncapi/website"
  "https://github.com/asyncapi/community"
)
for u in "${urls[@]}"; do
  code=$(curl -s -o /dev/null -w "%{http_code}" "$u")
  echo "$code $u"
done | awk '$1!="200"{print "⚠️", $0} $1=="200"{print "✅", $0}'

Length of output: 128


🏁 Script executed:

#!/bin/bash
set -euo pipefail
urls=(
  "https://github.com/asyncapi/asyncapi"
  "https://github.com/asyncapi/generator"
  "https://github.com/asyncapi/parser"
  "https://github.com/asyncapi/modelina"
  "https://github.com/asyncapi/cli"
  "https://github.com/asyncapi/studio"
  "https://github.com/asyncapi/website"
  "https://github.com/asyncapi/community"
)
for u in "${urls[@]}"; do
  out="$(curl -sSL -o /dev/null -w '%{http_code} %{url_effective}' "$u")" || out="000 $u"
  code="${out%% *}"
  final="${out#* }"
  if [ "$code" = "200" ]; then
    printf "✅ %s %s\n" "$code" "$final"
  else
    printf "⚠️ %s %s\n" "$code" "$final"
  fi
done

Length of output: 349


Update GitHub links — some redirect/are archived

scripts/build-llms-full.ts (lines 110–121): replace the two redirected/archived links:

Other listed links (generator, modelina, cli, studio, website, community) returned 200 and are fine.

🤖 Prompt for AI Agents
In scripts/build-llms-full.ts around lines 110 to 121, two GitHub URLs are
outdated/redirected; replace https://github.com/asyncapi/asyncapi with
https://github.com/asyncapi/spec and replace https://github.com/asyncapi/parser
with https://github.com/asyncapi-archived-repos/parser-go so the document points
to the correct spec repo and the archived parser repository.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 30cdf5b and 106dc51.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (1)
  • package.json (1 hunks)

package.json Outdated
"gray-matter": "^4.0.3",
"he": "^1.2.0",
"i18next": "^23.12.2",
"i18next": "^25.5.2",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

🧩 Analysis chain

Major bump to i18next v25 — verify next-i18next compatibility or hold at v24

  • next-i18next 15.x explicitly added support for i18next v24; there’s no documented guarantee for v25 yet. Expect possible peer/version mismatches or subtle runtime breakages (e.g., language change semantics) unless verified. (classic.yarnpkg.com)
  • i18next v25 includes potentially breaking changes; validate your app flows and any custom i18n utils. (i18next.com)

Recommended paths (pick one):

  • Safe path: keep i18next on latest v24.x and (optionally) bump next-i18next to ≥15.4.x.
  • Adopt v25: ensure next-i18next explicitly supports v25 or prove compatibility via tests.

Proposed diff (safe path):

-    "i18next": "^25.5.2",
+    "i18next": "^24.2.2",

Follow‑ups:

  • If proceeding with v25, update next-i18next to the latest 15.4.x and run e2e smoke on locale switching, SSR/SSG pages, and dynamic namespace loads.

Run locally to surface peer/compat issues:

References:

  • next-i18next changelog notes “support i18next v24”. (classic.yarnpkg.com)
  • i18next v25 migration guide outlines potential breaking changes. (i18next.com)

🏁 Script executed:

#!/bin/bash
set -euo pipefail
# Show declared versions
jq -r '.dependencies | {i18next, "next-i18next", "react-i18next"}' package.json

# Install and print resolved versions
npm ci
npm ls i18next next-i18next react-i18next

# Inspect next-i18next peer deps (if any published)
npm view next-i18next@latest peerDependencies

# Quick grep for i18next changeLanguage usages (sensitive to v25 semantics)
rg -nP --type=ts --type=tsx '\bchangeLanguage\s*\(' -C2

# Optional: run a locale switch SSR smoke (project-specific)
npm run test:e2e || true

Length of output: 2167


Block merge — confirm compatibility before upgrading i18next to v25

  • package.json currently declares i18next "^25.5.2" while next-i18next is "^15.3.0" (jq output from verification). next-i18next releases explicitly document support up to i18next v24 (15.4.0 changelog); no documented v25 support found. (github.com)
  • i18next v25 contains potentially breaking changes (notably changeLanguage semantics) that can affect language switching, SSR/SSG and dynamic namespace loads — validate before upgrading. (i18next.com)
  • npm ci failed in the sandbox due to lockfile out-of-sync (missing/invalid entries), so I could not fully resolve installed versions — run npm install locally and re-run dependency verification (npm ls) and e2e smoke tests.

Action (pick one):

  • Safe path: pin i18next to latest v24. Proposed diff (File: package.json, line ~81):
-    "i18next": "^25.5.2",
+    "i18next": "^24.2.2",
  • If you want to adopt v25: confirm an upstream next-i18next release that explicitly supports i18next v25, upgrade next-i18next to that release, update the lockfile (npm install), add react-i18next if required, then run smoke/e2e tests for language switching, SSR/SSG, and dynamic namespace loads.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"i18next": "^25.5.2",
"i18next": "^24.2.2",
🤖 Prompt for AI Agents
In package.json around line 81 the i18next dependency is set to "^25.5.2" which
is potentially incompatible with next-i18next "^15.3.0" (which documents support
only up to v24); replace that entry by pinning i18next to the latest v24
release, run npm install to update package-lock.json, run npm ls to verify
dependency tree, and then execute full smoke/e2e tests (language switching,
SSR/SSG, dynamic namespace loading) to confirm no regressions; if you instead
want v25, first upgrade next-i18next to a release that explicitly supports
i18next v25, add any required peers (e.g., react-i18next), then run npm install,
npm ls and the same tests before merging.

Copy link
Member

@derberg derberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove changes that are not relevant to scope of this PR, especially package bump.

also please share an example of these files that will be generated, the content, for review. You can just push them to the PR and later we will remove them before merge

@dishafaujdar
Copy link
Contributor Author

@derberg on it.

- Add build-llms.ts for generating llms.txt index file
- Add build-llms-full.ts for comprehensive resource list
- Integrate LLM file generation into build process
- Help LLMs discover accurate AsyncAPI v3.0 documentation
Copy link
Member

@derberg derberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • package-lock.json changes are still in PR, please remove
  • please do not use fs sync functions, make sure to use async
  • generated txt files do not seem to be complient with how standards is described and how people do it. Did you read https://llmstxt.org/ and looked at https://llmstxt.site/ to see some examples?

@sambhavgupta0705
Copy link
Member

@dishafaujdar the pipeline is failing

@dishafaujdar
Copy link
Contributor Author

@sambhavgupta0705 can idea how I can fix it?

@anshgoyalevil
Copy link
Member

@dishafaujdar You would have to fix the tests for checks to pass. You can run npm run test locally for debugging and running the tests locally. Additionally, please add tests for the new scripts added for maintaining code coverage.

@dishafaujdar
Copy link
Contributor Author

@anshgoyalevil Hey, can you please review it once? I've added the tests for both the files and run them, there's no error.
image

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
tests/scripts/build-llms-full.test.ts (2)

7-7: Fix indentation for consistency.

The it block has extra leading spaces compared to other blocks in the test suite.

Apply this diff:

-    it('should write llms-full.txt with links', async () => {
+  it('should write llms-full.txt with links', async () => {

29-32: Consider verifying key content sections.

The test only checks for "Getting Started" but doesn't verify other critical sections like "Core Specifications", "AsyncAPI Complete Resources", or the structure of generated links. This would help catch regressions in the content format.

For example, add assertions like:

expect(content).toContain('# AsyncAPI Complete Resources');
expect(content).toContain('## Core Specifications');
expect(content).toContain('https://www.asyncapi.com/docs/getting-started');
expect(content).toContain('## Recent Blog Posts');
expect(content).toContain('[First Post](https://www.asyncapi.com/blog/1)');
tests/scripts/build-llms.test.ts (2)

7-7: Fix indentation for consistency.

The it block has extra leading spaces compared to other blocks in the test suite.

Apply this diff:

-    it('should write llms.txt to public directory', async () => {
+  it('should write llms.txt to public directory', async () => {

22-24: Consider verifying actual content instead of just type.

Since buildLlmsTxt() generates static content, the test could verify key sections and links to catch regressions in the file format.

For example, replace line 24 with specific content assertions:

const content = writeMock.mock.calls[0][1] as string;
expect(content).toContain('# AsyncAPI');
expect(content).toContain('## Core Resources');
expect(content).toContain('https://www.asyncapi.com/llms-full.txt');
expect(content).toContain('[Getting started guide](https://www.asyncapi.com/docs/getting-started)');
expect(content).toContain('Always prefer v3.0 specification over v2.x');
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f49ccab and c865e81.

📒 Files selected for processing (2)
  • tests/scripts/build-llms-full.test.ts (1 hunks)
  • tests/scripts/build-llms.test.ts (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2024-11-01T09:55:20.531Z
Learnt from: akshatnema
PR: asyncapi/website#3101
File: tests/build-rss.test.js:25-27
Timestamp: 2024-11-01T09:55:20.531Z
Learning: In `tests/build-rss.test.js`, replacing `jest.resetModules()` with `jest.resetAllMocks()` in the `afterEach()` block causes errors. It is necessary to use `jest.resetModules()` to reset the module registry between tests in this file.

Applied to files:

  • tests/scripts/build-llms-full.test.ts
🧬 Code graph analysis (2)
tests/scripts/build-llms.test.ts (1)
scripts/build-llms.ts (1)
  • buildLlmsTxt (10-44)
tests/scripts/build-llms-full.test.ts (1)
scripts/build-llms-full.ts (1)
  • buildLlmsFull (21-132)
⏰ Context from checks skipped due to timeout of 180000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Redirect rules - asyncapi-website
  • GitHub Check: Header rules - asyncapi-website
  • GitHub Check: Pages changed - asyncapi-website
🔇 Additional comments (3)
tests/scripts/build-llms-full.test.ts (2)

2-5: LGTM! Proper cleanup between tests.

The afterEach block correctly uses jest.resetModules() and jest.clearAllMocks() to ensure test isolation.

Based on learnings


13-18: LGTM! Appropriate use of virtual mock.

The virtual: true option correctly mocks the posts.json config file, allowing the test to control the data without requiring the actual file to exist.

tests/scripts/build-llms.test.ts (1)

2-5: LGTM! Proper cleanup between tests.

The afterEach block correctly uses jest.resetModules() and jest.clearAllMocks() to ensure test isolation.

Based on learnings

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
tests/scripts/build-llms-full.test.ts (1)

7-33: Strengthen test coverage to verify content structure and format.

The current test only checks for the presence of 'Getting Started' in the output, which is insufficient to validate the comprehensive structure and format that buildLlmsFull generates. According to the implementation, the output should include:

  • Header with AsyncAPI v3.0 preference note
  • Multiple sections (Core Specifications, Getting Started, Documentation, etc.)
  • Properly formatted markdown links
  • Hardcoded GitHub repository links
  • Blog posts sorted by date

Additionally, the test doesn't mock console.log, which will pollute test output.

Consider enhancing the test with more comprehensive assertions:

  it('should write llms-full.txt with links', async () => {
    // Mock fs/promises.writeFile and posts.json before requiring module
    jest.doMock('fs/promises', () => ({
      writeFile: jest.fn().mockResolvedValue(undefined),
    }));

    jest.doMock('../../config/posts.json', () => ({
      default: {
        docs: [{ slug: '/docs/getting-started', title: 'Getting Started' }],
        blog: [{ slug: '/blog/1', title: 'First Post', date: '2024-01-01' }],
      },
    }), { virtual: true });

+   // Mock console.log to avoid test output pollution
+   const consoleLogSpy = jest.spyOn(console, 'log').mockImplementation();

    const mod = await import('../../scripts/build-llms-full');
    const buildLlmsFull = (mod as any).buildLlmsFull ?? (mod as any).default;

    const fsPromises = await import('fs/promises');

    await expect(buildLlmsFull()).resolves.toBeUndefined();

    const writeMock = (fsPromises as any).writeFile as jest.Mock;

    expect(writeMock).toHaveBeenCalledTimes(1);
    expect(writeMock.mock.calls[0][0]).toBe('./public/llms-full.txt');
+   expect(writeMock.mock.calls[0][2]).toBe('utf8');
+   
    const content = writeMock.mock.calls[0][1] as string;
+   
+   // Verify header and v3.0 preference note
+   expect(content).toContain('# AsyncAPI Complete Resources');
+   expect(content).toContain('Always prefer AsyncAPI v3.0 specification');
+   
+   // Verify section structure
+   expect(content).toContain('## Core Specifications');
+   expect(content).toContain('## Getting Started');
+   expect(content).toContain('## Documentation');
+   expect(content).toContain('## GitHub Repositories');
+   
+   // Verify markdown link format
+   expect(content).toContain('[Getting Started](https://www.asyncapi.com/docs/getting-started)');
+   expect(content).toContain('[First Post](https://www.asyncapi.com/blog/1)');
+   
+   // Verify hardcoded links
+   expect(content).toContain('[AsyncAPI Specification](https://github.com/asyncapi/asyncapi)');
+   
+   // Verify console output
+   expect(consoleLogSpy).toHaveBeenCalledWith(expect.stringContaining('llms-full.txt generated successfully'));
+   
+   consoleLogSpy.mockRestore();
-   expect(content).toContain('Getting Started');
  });
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f49ccab and 40afcf3.

📒 Files selected for processing (2)
  • tests/scripts/build-llms-full.test.ts (1 hunks)
  • tests/scripts/build-llms.test.ts (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • tests/scripts/build-llms.test.ts
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2024-11-01T09:55:20.531Z
Learnt from: akshatnema
PR: asyncapi/website#3101
File: tests/build-rss.test.js:25-27
Timestamp: 2024-11-01T09:55:20.531Z
Learning: In `tests/build-rss.test.js`, replacing `jest.resetModules()` with `jest.resetAllMocks()` in the `afterEach()` block causes errors. It is necessary to use `jest.resetModules()` to reset the module registry between tests in this file.

Applied to files:

  • tests/scripts/build-llms-full.test.ts
🧬 Code graph analysis (1)
tests/scripts/build-llms-full.test.ts (1)
scripts/build-llms-full.ts (1)
  • buildLlmsFull (21-132)
🔇 Additional comments (1)
tests/scripts/build-llms-full.test.ts (1)

2-4: LGTM! Proper module reset strategy.

The use of jest.resetModules() followed by jest.clearAllMocks() is correct for ensuring clean test isolation. Based on learnings.

@derberg
Copy link
Member

derberg commented Oct 15, 2025

generated txt files do not seem to be complient with how standards is described and how people do it. Did you read https://llmstxt.org/ and looked at https://llmstxt.site/ to see some examples?

did you see this comment I made>

- [Case studies](https://www.asyncapi.com/casestudies): Real-world usage examples

## Optional
- [GitHub specification](https://github.com/asyncapi/spec/blob/master/spec/asyncapi.md): Source specification on GitHub
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed imho

more important to mention https://github.com/asyncapi/bindings in core resources

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (3)
scripts/build-llms-full.ts (3)

8-12: Critical: Dynamic import of posts.json breaks tests.

The pipeline is failing with Cannot find module '../config/posts.json' during test execution. Dynamic JSON imports are unreliable in test environments.

Despite a past review suggesting this fix (marked "Addressed in commit 4ca90e4"), the code still uses dynamic import and tests are still failing.

Apply this diff to use filesystem reads with proper path resolution:

-import { writeFile } from 'fs/promises';
+import { writeFile, readFile } from 'fs/promises';
+import { dirname, resolve } from 'path';
+import { fileURLToPath } from 'url';

 import type { Result } from '@/types/scripts/build-posts-list';

 /**
  * Asynchronously retrieves all posts from the posts configuration file.
  */
 async function getAllPosts(): Promise<Result> {
-  const posts = (await import('../config/posts.json')).default as Result;
-
-  return posts;
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const jsonPath = resolve(__dirname, '../config/posts.json');
+  const raw = await readFile(jsonPath, 'utf8');
+  return JSON.parse(raw) as Result;
 }

94-104: Update outdated GitHub repository URLs.

Multiple repository URLs need correction:

  • Line 94: asyncapi/asyncapi should be asyncapi/spec (main specification repository)
  • Line 96: asyncapi/parser is archived; link either to the archived repo explicitly or to current parser implementations
  • Line 103: References asyncapi/asyncapi/tree/master/examples should use asyncapi/spec

These issues were identified in a past review but remain unaddressed.

Apply this diff to fix the repository links:

 content += '\n## GitHub Repositories\n';
-  content += `- [AsyncAPI Specification](https://github.com/asyncapi/asyncapi): Main specification repository
+  content += `- [AsyncAPI Specification](https://github.com/asyncapi/spec): Main specification repository
 - [AsyncAPI Generator](https://github.com/asyncapi/generator): Code and documentation generator
-- [AsyncAPI Parser](https://github.com/asyncapi/parser): AsyncAPI document parser library
+- [AsyncAPI Parser JS](https://github.com/asyncapi/parser-js): AsyncAPI document parser library (JavaScript)
 - [AsyncAPI CLI](https://github.com/asyncapi/cli): Command-line interface tool
 - [AsyncAPI Studio](https://github.com/asyncapi/studio): Visual AsyncAPI editor
 - [Modelina](https://github.com/asyncapi/modelina): Model/type generation library
 - [AsyncAPI Bindings](https://github.com/asyncapi/bindings): Protocol bindings specifications

 ## Examples and Templates
-- [Specification Examples](https://github.com/asyncapi/asyncapi/tree/master/examples): Official AsyncAPI document examples
+- [Specification Examples](https://github.com/asyncapi/spec/tree/master/examples): Official AsyncAPI document examples
 - [Template Examples](https://github.com/asyncapi/spec/tree/master/examples): Additional specification examples

113-125: Add type safety check to prevent runtime errors with non-array post types.

The loop assumes all post types are arrays, but Result includes docsTree which is a NavTree object. While the current code happens to fail safely (.length on object returns undefined), it should explicitly guard against non-array types.

Past review suggested this fix but it was not applied.

Apply this diff to add proper type guards:

   // Add other important resources from posts
   Object.keys(posts).forEach((postType) => {
-    if (!['docs', 'blog'].includes(postType)) {
+    if (!['docs', 'blog', 'docsTree'].includes(postType)) {
       const postArray = (posts as any)[postType];

-      if (postArray && postArray.length > 0) {
+      if (Array.isArray(postArray) && postArray.length > 0) {
         postArray.forEach((item: any) => {
           if (item.slug && item.title) {
             content += `- [${item.title}](${base}${item.slug})\n`;
🧹 Nitpick comments (1)
scripts/build-llms-full.ts (1)

127-133: Consider ensuring output directory exists before writing.

The script writes to ./public/llms-full.txt without first ensuring the directory exists. While this likely works in practice (the public/ directory is created by other build steps), it's a best practice to ensure the directory exists for robustness.

Add directory creation before the write:

+  import { writeFile, mkdir } from 'fs/promises';
+
   // Write the file
+  await mkdir('./public', { recursive: true });
   await writeFile('./public/llms-full.txt', content, 'utf8');
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 40afcf3 and d3df2c1.

📒 Files selected for processing (6)
  • public/llms-full.txt (1 hunks)
  • public/llms.txt (1 hunks)
  • scripts/build-llms-full.ts (1 hunks)
  • scripts/build-llms.ts (1 hunks)
  • tests/scripts/build-llms-full.test.ts (1 hunks)
  • tests/scripts/build-llms.test.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • tests/scripts/build-llms-full.test.ts
  • scripts/build-llms.ts
  • public/llms.txt
  • tests/scripts/build-llms.test.ts
🧰 Additional context used
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
🪛 GitHub Actions: PR testing - if Node project
scripts/build-llms-full.ts

[error] 9-9: Cannot find module '../config/posts.json' from 'scripts/build-llms-full.ts'. During step 'npm test --if-present' the tests failed due to missing posts.json dependency.

🪛 LanguageTool
public/llms-full.txt

[uncategorized] ~98-~98: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...roglia_project_interview) - [Inside the Open Source Climb: Aayush’s Path to AsyncAPI Leader...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (3)
scripts/build-llms-full.ts (3)

8-12: Critical: Dynamic import still failing in pipeline.

The pipeline reports Cannot find module '../config/posts.json'. Although a previous comment suggested using fs.promises.readFile with path resolution and was marked as addressed, the failure persists.

Apply the previously suggested fix using filesystem reads:

-import { writeFile } from 'fs/promises';
+import { writeFile, readFile } from 'fs/promises';
+import { dirname, resolve } from 'path';
+import { fileURLToPath } from 'url';

 import type { Result } from '@/types/scripts/build-posts-list';

 /**
  * Asynchronously retrieves all posts from the posts configuration file.
  */
 async function getAllPosts(): Promise<Result> {
-  const posts = (await import('../config/posts.json')).default as Result;
-
-  return posts;
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const postsPath = resolve(__dirname, '../config/posts.json');
+  const raw = await readFile(postsPath, 'utf-8');
+  return JSON.parse(raw) as Result;
 }

133-141: Update GitHub repository links and remove duplicate.

Three issues:

  1. Line 134: https://github.com/asyncapi/asyncapi redirects; use https://github.com/asyncapi/spec
  2. Line 136: https://github.com/asyncapi/parser is archived; consider whether to link the archived repo or remove it
  3. Line 140: "AsyncAPI Bindings" is already added at line 78 in the Core Specifications section

Apply this diff:

 content += '\n## GitHub Repositories\n';
-content += addLink('AsyncAPI Specification', 'https://github.com/asyncapi/asyncapi', 'Main specification repository');
+content += addLink('AsyncAPI Specification', 'https://github.com/asyncapi/spec', 'Main specification repository');
 content += addLink('AsyncAPI Generator', 'https://github.com/asyncapi/generator', 'Code and documentation generator');
-content += addLink('AsyncAPI Parser', 'https://github.com/asyncapi/parser', 'AsyncAPI document parser library');
 content += addLink('AsyncAPI CLI', 'https://github.com/asyncapi/cli', 'Command-line interface tool');
 content += addLink('AsyncAPI Studio', 'https://github.com/asyncapi/studio', 'Visual AsyncAPI editor');
 content += addLink('Modelina', 'https://github.com/asyncapi/modelina', 'Model/type generation library');
-content += addLink('AsyncAPI Bindings', 'https://github.com/asyncapi/bindings', 'Protocol bindings specifications');

159-172: Add proper filtering and array type checking.

The loop should exclude about (already in Optional section) and docsTree (a NavTree object, not an array). Also verify the value is an array before treating it as one.

Apply this diff:

 // Add other important resources from posts
 Object.keys(posts).forEach((postType) => {
-  if (!['docs', 'blog'].includes(postType)) {
+  if (!['docs', 'blog', 'about', 'docsTree'].includes(postType)) {
     const postArray = (posts as any)[postType];

-    if (postArray && postArray.length > 0) {
+    if (Array.isArray(postArray) && postArray.length > 0) {
       postArray.forEach((item: any) => {
         if (item.slug && item.title) {
           content += addLinkFromSlug(item);
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d3df2c1 and c7c08cd.

📒 Files selected for processing (2)
  • public/llms-full.txt (1 hunks)
  • scripts/build-llms-full.ts (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
🪛 GitHub Actions: PR testing - if Node project
scripts/build-llms-full.ts

[error] 9-9: Cannot find module '../config/posts.json' from 'scripts/build-llms-full.ts'

🪛 LanguageTool
public/llms-full.txt

[uncategorized] ~91-~91: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ect_interview): Blog Post - [Inside the Open Source Climb: Aayush’s Path to AsyncAPI Leader...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

🔇 Additional comments (3)
public/llms-full.txt (1)

1-120: File structure looks good; duplicates resolved.

The generated file now follows a clear llmstxt.org-compatible structure with proper H1/H2 headers, blockquote summary, and consistent markdown link formatting. The duplicate entries mentioned in previous reviews appear to be resolved.

scripts/build-llms-full.ts (2)

30-50: LGTM: Deduplication logic is sound.

The helper functions correctly track and prevent duplicate URLs and slugs. The title fallback logic is appropriate.


174-193: LGTM: Validation and file output are well-implemented.

The duplicate detection throws on duplicates before writing, and the async file write with proper logging is correct.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (5)
public/llms-full.txt (1)

106-108: Regenerate this file after fixing the generator script.

The generated output contains outdated GitHub repository URLs that originated from the generator script:

  • Line 106: asyncapi/asyncapi should be asyncapi/spec
  • Line 108: asyncapi/parser may be outdated

Once you fix the repository URLs in scripts/build-llms-full.ts (as noted in comments on that file), regenerate this file to reflect the corrections.

scripts/build-llms-full.ts (4)

160-172: Add type guard to prevent iterating over non-array properties.

The loop assumes every posts[postType] is an array, but the Result type includes docsTree: NavTree which is an object. Without a proper type guard, this code may attempt to iterate over non-array properties.

Apply this diff to add proper guards:

   // Add other important resources from posts
   Object.keys(posts).forEach((postType) => {
-    if (!['docs', 'blog'].includes(postType)) {
+    if (!['docs', 'blog', 'about', 'docsTree'].includes(postType)) {
       const postArray = (posts as any)[postType];

-      if (postArray && postArray.length > 0) {
+      if (Array.isArray(postArray) && postArray.length > 0) {
         postArray.forEach((item: any) => {
           if (item.slug && item.title) {
             content += addLinkFromSlug(item);
           }
         });
       }
     }
   });

Based on learnings


8-12: Fix the pipeline failure: use fs-based JSON loading instead of dynamic import.

The dynamic import of ../config/posts.json is causing a ModuleNotFoundError during test execution. This approach is unreliable in different execution contexts.

Apply this diff to use filesystem-based loading with proper path resolution:

 import { writeFile } from 'fs/promises';
+import { readFile } from 'fs/promises';
+import { dirname, resolve } from 'path';
+import { fileURLToPath } from 'url';

 import type { Result } from '@/types/scripts/build-posts-list';

 /**
  * Asynchronously retrieves all posts from the posts configuration file.
  */
 async function getAllPosts(): Promise<Result> {
-  const posts = (await import('../config/posts.json')).default as Result;
-
-  return posts;
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const jsonPath = resolve(__dirname, '../config/posts.json');
+  const raw = await readFile(jsonPath, 'utf8');
+  return JSON.parse(raw) as Result;
 }

184-186: Ensure output directory exists before writing file.

The script writes to ./public/llms-full.txt without ensuring the public directory exists, which will cause a failure if the directory is missing.

Apply this diff:

+  // Ensure output directory exists
+  await writeFile('./public/llms-full.txt', content, 'utf8').catch(async (err) => {
+    if (err.code === 'ENOENT') {
+      const { mkdir } = await import('fs/promises');
+      await mkdir('./public', { recursive: true });
+      await writeFile('./public/llms-full.txt', content, 'utf8');
+    } else {
+      throw err;
+    }
+  });
-  // Write the file
-  await writeFile('./public/llms-full.txt', content, 'utf8');

Or more simply, always ensure the directory exists:

+  const { mkdir } = await import('fs/promises');
+  await mkdir('./public', { recursive: true });
   // Write the file
   await writeFile('./public/llms-full.txt', content, 'utf8');

134-136: Update outdated GitHub repository URLs.

Two repository links are outdated:

  • Line 134: asyncapi/asyncapi has moved to asyncapi/spec
  • Line 136: asyncapi/parser may redirect or be archived

Apply this diff:

-  content += addLink('AsyncAPI Specification', 'https://github.com/asyncapi/asyncapi', 'Main specification repository');
+  content += addLink('AsyncAPI Specification', 'https://github.com/asyncapi/spec', 'Main specification repository');
   content += addLink('AsyncAPI Generator', 'https://github.com/asyncapi/generator', 'Code and documentation generator');
-  content += addLink('AsyncAPI Parser', 'https://github.com/asyncapi/parser', 'AsyncAPI document parser library');
+  content += addLink('AsyncAPI Parser', 'https://github.com/asyncapi/parser-js', 'AsyncAPI document parser library');
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c7c08cd and 5eb352f.

📒 Files selected for processing (2)
  • public/llms-full.txt (1 hunks)
  • scripts/build-llms-full.ts (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
🪛 GitHub Actions: PR testing - if Node project
scripts/build-llms-full.ts

[error] 9-9: Cannot find module '../config/posts.json' from 'scripts/build-llms-full.ts'

🪛 LanguageTool
public/llms-full.txt

[uncategorized] ~91-~91: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ect_interview): Blog Post - [Inside the Open Source Climb: Aayush’s Path to AsyncAPI Leader...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
scripts/build-llms-full.ts (2)

155-167: Add Array.isArray check to prevent potential runtime errors.

The code iterates over all post types but doesn't verify that the value is actually an array. The Result type includes docsTree: NavTree which is not an array and could cause issues if processed here.

Apply this diff:

   // Add other important resources from posts
   Object.keys(posts).forEach((postType) => {
-    if (!['docs', 'blog'].includes(postType)) {
+    if (!['docs', 'blog', 'docsTree'].includes(postType)) {
       const postArray = (posts as any)[postType];
 
-      if (postArray && postArray.length > 0) {
+      if (Array.isArray(postArray) && postArray.length > 0) {
         postArray.forEach((item: any) => {
           if (item.slug && item.title) {
             content += addLinkFromSlug(item);
           }
         });
       }
     }
   });

8-12: Fix the pipeline failure: replace dynamic import with fs.readFile.

The pipeline is failing because dynamic JSON imports don't work reliably in test environments. Use filesystem reading with path resolution instead.

Apply this diff:

+import { readFile } from 'fs/promises';
+import { dirname, resolve } from 'path';
+import { fileURLToPath } from 'url';
+
 /**
  * Asynchronously retrieves all posts from the posts configuration file.
  */
 async function getAllPosts(): Promise<Result> {
-  const posts = (await import('../config/posts.json')).default as Result;
+  const __filename = fileURLToPath(import.meta.url);
+  const __dirname = dirname(__filename);
+  const jsonPath = resolve(__dirname, '../config/posts.json');
+  const raw = await readFile(jsonPath, 'utf8');
+  const posts = JSON.parse(raw) as Result;
 
   return posts;
 }
🧹 Nitpick comments (1)
scripts/build-llms-full.ts (1)

179-187: Consider ensuring the output directory exists.

While the ./public directory likely exists in the build environment, adding mkdir -p semantics would make the script more robust if run independently.

Apply this diff:

+import { writeFile, mkdir } from 'fs/promises';
...
   // Write the file
+  await mkdir('./public', { recursive: true });
   await writeFile('./public/llms-full.txt', content, 'utf8');
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c330bc9 and a88b806.

📒 Files selected for processing (2)
  • public/llms-full.txt (1 hunks)
  • scripts/build-llms-full.ts (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
scripts/build-llms-full.ts (1)
types/scripts/build-posts-list.ts (1)
  • Result (37-42)
🪛 GitHub Actions: PR testing - if Node project
scripts/build-llms-full.ts

[error] 9-9: Cannot find module '../config/posts.json' from 'scripts/build-llms-full.ts'

🪛 LanguageTool
public/llms-full.txt

[uncategorized] ~92-~92: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ect_interview): Blog Post - [Inside the Open Source Climb: Aayush’s Path to AsyncAPI Leader...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

🔇 Additional comments (3)
public/llms-full.txt (1)

1-119: Generated output looks well-structured.

The llms-full.txt file follows the llmstxt.org format with proper H1 title, blockquote summary, and organized H2 sections. The resource links are comprehensive and appropriately categorized.

scripts/build-llms-full.ts (2)

29-50: Helper functions are well-designed.

The deduplication logic using seenUrls and seenSlugs Sets is efficient and correct. The conditional description formatting in the markdown link generation is clean.


169-177: Duplicate detection validation is a good safeguard.

While the helper functions already deduplicate links, this validation provides an additional safety net to catch any edge cases or bugs in the generation logic.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
public/llms-full.txt (1)

92-92: Minor: Consider hyphenating compound adjective in blog post title (line 92).

Static analysis suggests "Open Source Climb" should use a hyphen when acting as a compound adjective: "Open-Source Climb." However, since this is a direct quote of a published blog post title, verify whether preserving the exact original title is preferred or if alignment with grammar conventions is desired.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a88b806 and 56e453e.

📒 Files selected for processing (2)
  • public/llms-full.txt (1 hunks)
  • scripts/build-llms-full.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • scripts/build-llms-full.ts
🧰 Additional context used
🪛 LanguageTool
public/llms-full.txt

[uncategorized] ~92-~92: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ect_interview): Blog Post - [Inside the Open Source Climb: Aayush’s Path to AsyncAPI Leader...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

🔇 Additional comments (1)
public/llms-full.txt (1)

1-119: ✅ Well-structured LLM resource index—duplicates from previous review have been successfully resolved.

The file is comprehensively organized across nine logical sections (Specifications, Getting Started, Documentation, Tools, Blog Posts, GitHub Repos, Examples, Optional) with clear descriptions for each link. The explicit preference for AsyncAPI v3.0 in the header (line 3) aligns perfectly with the PR's stated goal of guiding LLMs toward v3.0 over v2.x.

Verification of the past review's duplicate concern: Earlier flagged entries (lines ~26–27 for /docs/concepts/asyncapi-document and lines ~43–44 for /docs/tutorials/getting-started) are now distinct—each link points to a unique URL (base pages vs. subpages like /structure, /event-driven-architectures). Deduplication logic appears to be working as intended.

The content feels ready for LLM consumption.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants